The Performance of Work Stealing in Multiprogrammed Environments

As small-scale, shared-memory multiprocessors make their way onto desktops, the high-performance parallel applications that run on these machines will have to live alongside other applications, such as editors and web browsers. Unfortunately, unless parallel applications are coscheduled [4] or subject to process control [2], they display poor performance in such multiprogrammed environments [2]. As an alternative to coscheduling or process control, we investigate the use of dynamic, user-level, thread scheduling. In particular, we show that a non-blocking [3] implementation of the work-stealing thread-scheduling algorithm [1] achieves efficient performance even when the number of available processors grows and shrinks over time. All of the experiments in this paper were run on a Sun Ultra Enterprise 5000 with 8 167-Mhz UltraSPARC processors running Solaris 2.5.1, with no modifications. The work-stealing thread scheduler studied in this paper has been implemented in a C++ threads library called Hood, and our applications are coded in C++ using Hood. Hood is implemented on top of the Solaris threads library. Traditionally, multithreaded applications use static partitioning and display poor performance when run in a multiprogrammed environment. Such a program creates some number P of (lightweight) processes (also known as kernel threads), and each process performs a 1=P fraction of the total work. Let T1 denote the work of the computation, which we define as the execution time with P = 1 process running on 1 dedicated processor. With P > 1 processes running on P dedicated processors, if the overhead of creating and synchronizing these processes is small compared to the T1=P work per process, then the execution time will be TP = T1=P , thereby giving a speedup of T1=TP = P . In a multiprogrammed environment, however, we might find that the actual number PA of processors on which the program runs is smaller than the number P of processes. In this case, we can still hope to achieve an execution time of TP = T1=PA, thereby giving a speedup of T1=TP = PA. Unfortunately, as Figure 1(a) shows, this hope is not being realized. This figure shows the measured speedup of several statically partitioned applications for different numbers P of processes.